Method of tracking position and velocity of objects&#39; borders in two or three dimensional digital images, particularly in echographic images

ABSTRACT

A method of tracking position and velocity of objects&#39; borders in two or three dimensional digital images, particularly in echographic images. A sequence is acquired of at least two consecutive ultrasound image frames of a moving tissue or a moving object. The frames are separated by a certain time interval. Reference points define a border of the moving tissue or object and the border is automatically tracked by estimating the position of the reference points in following image frames on the basis of the ultrasound image data of the acquired sequence of image frames.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims the benefit of European PatentApplication Serial No. 03425639.6 filed on Sep. 30, 2003, which ishereby incorporated by reference in its entirety.

BACKGROUND OF THE INVENTION

The invention relates to a method of tracking position and velocity ofobjects' borders in two or three dimensional echographic images.

The automatic detection of borders and their rate of displacement is afundamental topic in image analysis. The ability to automatically detectborders and their velocity allows easy recognition of objects, and animproved understanding of their function. In medical imaging theautomatic recognition of an organ would eventually facilitate theextraction of objective measurements and automate some diagnosticprocesses.

A typical example is given by echocardiography in the imaging of theleft ventricle. The possibility of an automatic detection of theendocardial border would give objective measurement of the ventricularvolume. In particular, the extreme values (at proto-systole andtele-diastole) are commonly used for clinical diagnosis and the relativediagnostic index is calculated. In addition, the possibility tovisualize the border velocities allows an easier assessment of theregional dynamical properties.

In general, and certainly in echographic imaging, the development ofreliable methods for the automatic border detection is an extremelydifficult task that has not received a generally reliable solution. Infact, in clinical practice, borders are drawn manually by the operatorover the physiologically relevant frames of a sequence of images.

A strain rate analysis method in ultrasonic diagnostic imaging isdisclosed in WO 02/45587. According to this document strain rateanalysis is performed for ultrasonic images in which the spatialgradient of velocity is calculated in the direction of tissue motion.Strain rate is calculated for cardiac ultrasound images in the directionof motion which, for myocardial images, may be either in the plane ofthe myocardium or across the myocardium. Strain rate information iscalculated for a sequence of images of a heart cycle and displayed foran automatically drawn border such as the endocardial border over thefull heart cycle. The spatial gradient of velocity used for determiningthe strain and the displacements of the borders from one frame to asuccessive frame in a sequence of frames uses Doppler Tissue Imaging, orDTI. This technique allows the measurement of tissue velocity over allpoints in the ventricular wall. The measurement of velocity itselfprovides direct information about the wall motion and helps to uncoverabnormalities not immediately observable from the visualization inB-mode imaging. The velocity measurement contains information aboutrigid body displacement, shear, and contraction/distension, the latterbeing immediately related to the myocardial activity. Post processing ofthe DTI velocity data allows the evaluation of additional quantities,namely strain-rate and strain, that are strictly related to the regionalfunction. Segmental strain gives a direct evaluation of the degree ofcontractility of the myocardium during systole, as well as of itsrelaxation during ventricular filling.

Nevertheless, DTI suffers from a few drawbacks consisting in limitationsof the technique. The evaluation of velocity, and to a greater degree,rate and strain, requires a higher frame rate with respect to B-modeimaging because velocity is a more rapidly varying function than B-modedisplacement. A Doppler signal requires additional processing withrespect to a simple echo.

Doppler tissue imaging suffers further of an intrinsic limitation due tothe fact that only the component of velocity along a scanline can bemeasured. This limitation has several drawbacks. When tissue moves in adirection that is not aligned with the scanline, the Doppler velocitydoes not reflect the effective tissue kinematics. Only the component ofstrain and strain rate along the scanline can be evaluated correctly,giving a reduced view of the local deformation state. This limits theapplication of DTI to the anatomic sites that can be imagined alignedalong a scanline. In echocardiography this corresponds essentially tothe interventricular septum and to the lateral walls in apical view.

BRIEF SUMMARY

The present invention achieves the above mentioned aims with a method oftracking position and velocity of objects' borders in two or threedimensional digital images, particularly in echographic images, byacquiring a sequence of at least two consecutive ultrasound image framesof a moving tissue or a moving object, where the ultrasound image framesare separated by a certain time interval. A certain number of referencepoints of a border of a moving tissue or object at least on a firstimage frame of the sequence of image frames are defined. The border ofthe moving tissue or object in a following frame by is automaticallydetermined by determining the new position of the reference points ofthe border in the following image frame of the sequence of image frames.This determination is done by estimating the position of the referencepoints in the following image frame of the sequence of image frames onthe basis of the ultrasound image data of the acquired sequence of imageframes. The sequence of image frames acquired is a sequence ofconsecutive B-mode, grey scale ultrasound images. On a first frame aborder line is drawn either manually or by means of an automatic borderdetection algorithm. The border is defined by a trace of pixels of theimage frame coinciding with the border line. The original trace ofpixels coinciding with the manually or automatically drawn border lineis followed in time by searching the maximum likelihood of the trace ofpixels in the following image frame with the trace of pixels in thefirst or timely previous image frame of the sequence of image frames byanalyzing the image pixels in the neighborhood of the said trace ofpixels.

An object of the present invention is to provide a method for trackingposition and velocity of objects' borders in two or three dimensionaldigital images, particularly in echographic images, which overcomes thedrawbacks of the known tracking methods by reducing the load actuallyneeded for acquiring the usable ultrasound image data, and forevaluating ultrasound imaging data in order to track the displacement ofa border of a moving tissue or a moving object in a sequence ofconsecutive ultrasound image frames.

Another object of the present invention is to provide for a method whichcan lead to more reliable results in tracking the borders of a movingtissue or object on a sequence of consecutive ultrasound image frames.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 illustrates an echographic image of the left ventricle, in longaxis view (from the apex to the mitral plane), extracted from anechocardiographic recording (58 frames, 2 cycles) with an endocardialborder (white) and the instant border velocity (white arrows).

FIG. 2 illustrates an echographic image of the left ventricle, in shortaxis view (transversal section), extracted from an echocardiographicrecording (49 frames, 2 cycles) with a “closed” endocardial border(white) and the instant border velocity (white arrows).

FIG. 3 illustrates an echographic image of the left ventricle, in longaxis view, with the position of transmural cuts, passing from the edgepoints and normal to the transmittal plane, being indicated.

FIG. 4 illustrates a space-time representation, where space is along atransmural cut, of the echographic images sequence.

FIG. 5 illustrates an echographic image of the left ventricle, in longaxis view, during diastole (left) and systole (right), showing arescaled endocardial border and instantaneous transmural cuts on each ofthe original points.

FIG. 6 illustrates a schematic view of an example of cutting athree-dimensional dataset with two orthogonal planes, having one commondirection, to get two two-dimensional images.

FIG. 7 illustrates a schematic view of an example of cutting athree-dimensional dataset with transversal planes that are orthogonal tothe principal planes like those employed in FIG. 6.

FIG. 8 illustrates slices on a three-dimensional echographic dataset ofthe left heart.

FIG. 9 illustrates a horizontal slice on a three-dimensional echographicdataset of the left heart, showing starting points and a circular firstguess border.

DETAILED DESCRIPTION

For the purposes of promoting an understanding of the disclosure,reference will now be made to the embodiments illustrated in thedrawings and specific language will be used to describe the same. Itwill nevertheless be understood that no limitation of the scope of thedisclosure is thereby intended, such alterations and furthermodifications in the illustrated device and its use, and such furtherapplications of the principles of the disclosure as illustrated thereinbeing contemplated as would normally occur to one skilled in the art towhich the disclosure relates.

The following is an overview of the disclosure. A more detaileddescription follows with reference to the Figures.

According to one embodiment of the present invention, the tracking ofthe border line is carried out by defining a certain number of referencepoints on the manually or automatically drawn border line on the firstimage frame and by using the method of the so called transmural cuts.The method of transmural cuts is disclosed in greater detail in thedocument PCT/IT02/00114 filed on Feb. 27, 2002.

More precise transmural cuts consist in defining a line which crossesthe border line drawn and passing through one reference point. Aphysiologically appropriate direction can be chosen, which typically canbe the orthogonal direction to the border line at the reference point.This operation is made for each image frame of the sequence of framesand for each reference point chosen. The pixels taken along eachtransmural line in each of the image frames of the sequence of imageframes are placed in columns, with each column corresponding to oneframe of the sequence of images. In this way the evolution along atransmural cut can be represented in a two-dimensional space timerepresentation.

The above disclosed procedure is a reduction of a two dimensionalproblem applied to a two dimensional image, such as a B-mode ultrasoundimage, to a one dimensional problem as a M-mode image. The tracking ofthe border, i.e. of the trace of pixels, is carried out along thespace-time image using a cross-correlation procedure of the pixel columnin the space-time image corresponding to a first image frame with thepixel column in the space-time image corresponding to a successive imageframe of the sequence of image frames.

This technique can be applied to any kind of images in which thegeometry of the border line drawn does not require any kind of specialreference points to be tracked a priori such as, for example, closedborder lines as the border line of the cavity of a blood vessel in across-section image of the vessel. When the object imaged has particularstarting and ending points of a border which has a relevance asparticular reference points in the motion executed by the border-line ofthe object, for example in the case of the walls of the endoventricularcavity, then a preventive cycle must be carried out for optimallytracking the border-line of the object along the sequence of imageframes.

According to a further embodiment of the above mentioned method, whenthe object has few very representative points the general topology ofthe border line of the object imaged can be represented by tracking themotion of these few representative points prior to carry out thetracking of at least one or some of the reference points lying on themanually or automatically drawn border-line in each frame of thesequence of image frames. These representative points can be for examplethe starting and ending points of the border line when this is an openone. The representative reference points of the border of an imagedobject can be also suggested by the physiology when the imaged object isa particular tissue or organ, such as for example the left ventricle.Thus prior to carry out the tracking of some or all of the referencepoints chosen on the manually or automatically drawn border-line of theimaged object along the sequence of image frames the tracking of thisfew representative reference points is carried out.

The tracking of this few representative reference points is carried outin a identical way as the one disclosed above for the other referencepoints on the border-line drawn manually or automatically on the firstframe by using the method of transmural cuts for constructing space-timeimages of each of the few representative reference points anddetermining the displacement of these points in each of the frames ofthe sequence of image frames by means of cross-correlation between eachof the pixel columns with the successive pixel column corresponding tothe pixels along the transmural cut across the same representative pointin the different image frames of the sequence of image frames. Thedirection of the transmural cuts can be chosen as the orthogonaldirection.

According to a further feature, after having determined the displacementof the few representative reference points on the border-line of theimaged object in some or in all of the frames of the sequence of imageframes, the position and the displacement of the other reference pointson the border-lines at each image frame of the sequence of image framesare obtained by rescaling the originally drawn border-line in the firstimage frame in such a way to obtain in each image frame corresponding toa successive instant a topologically equivalent border line geometrywith respect to the original border line. Typically this results in atranslation of all points along the original border line. Thispreliminary resealing allows to keep the representative reference pointsalways in the proper position in all frames of the sequence of imageframes by rearranging the other reference points so that therepresentative reference points maintains the same meaning relatively tothe object in all frames of the sequence of image frames.

In a preferred embodiment, a sequence of at least two consecutiveultrasound image frames of a moving tissue or a moving object areacquired in which the ultrasound image frames are timely separated by acertain time interval. A border line is traced over one single firstframe either manually or with the help of an automatic border drawingalgorithm. The position displacements of one or more representativereference points are tracked over the entire sequence of consecutiveimage frames. The border line drawn on the first image frame is rescaledat least for some of the following image frames of the sequence of imageframes, according to the corresponding position tracked of therepresentative reference points. A certain number of further referencepoints are defined which are distributed along the border line on thefirst image frame and falling on the border line. The position of eachpoint is tracked independently from the others along the sequence ofimage frames. The position of the representative reference points and ofthe other reference points are tracked for each point independently ineach of the image frames of the sequence of image frames that define atransmural cut line consisting of a line which crosses the border linedrawn and passing through the reference point. The pixels taken alongeach transmural cut line in each of the image frames of the sequence ofimage frames are placed in columns, with each column corresponding toone frame of the sequence of images for representing the evolution alonga transmural cut line for all instants of time at once in atwo-dimensional space time representation. The tracking of the border,i.e., of the trace of pixels, along each transmural cut line is carriedout along the space-time image using a cross-correlation procedure ofthe pixel column in the space-time image corresponding to a first imageframe with the pixel column in the space-time image corresponding to asuccessive image frame of the sequence of image frames.

According to another embodiment, in both cases disclosed above whenimages are poor with a low signal-to-noise ratio, the space-timerepresentation along the transmural cuts can be built using a line forthe transmural cut with a thickness larger than that of a single pixeland by extracting the average value across such a thickness.

The above mentioned method can be further developed for carrying out asurface border tracking three dimensional imaging by acquiring asequence of three-dimensional ultrasound imaging data sets, eachthree-dimensional data set being acquired with a predetermined timeinterval from the previous one. At least a principal section plane ofeach three dimensional data set along one chosen direction is definedfor obtaining a sequence of two dimensional image frames along thesection plane. A border line of the object imaged is drawn eithermanually or automatically on the first two dimensional image frame ofthe sequence of two dimensional image frames taken along the saidsection plane. The previously described tracing step are repeated forthe two dimensional sequence of image frames, for each three dimensionaldata set of the sequence of three dimensional datasets defining apre-established number of further secondary section planes crossing theat least one, and preferably all, of the principal section planes, thesaid secondary section planes being spaced apart one from the otheralong a predetermined direction and dividing the object represented byeach three dimensional data set of the sequence of three-dimensionaldata sets in slices, for each secondary section plane in the sequence ofthree dimensional data sets constructing the corresponding sequence ofimage frames relative to the said secondary section plane, and for eachsequence of two dimensional image frames determining a guess border linein one single frame, by letting the border line passing across theintersection points with the principal section planes. The guess borderline is tracked by detecting a new border by applying the previouslydescribed steps and by substituting the time coordinate with the spatialcoordinate along the said guess border line by defining a certain numberof transmural cuts on the single image of the sequence of images alongthe guess border line. The pixels along the transmural cuts areidentified and placed along each transmural cuts side by side forconstructing a two dimensional image where the horizontal axis indicatesthe spatial coordinate along the guess border line. Cross correlationbetween each of the consecutive pixel columns in the said twodimensional image is calculated, thus tracking the border line in oneframe for each of the sequences of two dimensional image framescorresponding to each of the secondary section planes.

According to a further embodiment, one or more further principal sectionplanes can be defined. In a preferred embodiment two orthogonalprincipal section planes are chosen for carrying out the above mentionedmethod steps, the crossing line of the two principal section planesdefining a preferred direction of the planes. The direction can bechosen as suggested by the topological or functional feature of theobject imaged.

Considering ultrasound images of a biological tissue or of an organ suchas for example the heart the direction can be suggested by physiologicalreasons. For example this physiologically relevant direction can bechosen as the cut across a central vertical plane such as the ventricleaxis.

In order to better define the share of secondary section planes cuttingthe principal section planes, the method according to the presentinvention comprises the steps of defining bounds or limits for adistance range within which the share of the said secondary sectionplanes is defined.

Preferably a topological or physiological relevant direction is chosen,particularly the same direction defined for determining the principalsection planes, along which direction bounds are determined for the endsof a distance range within which the share of secondary section planesat least transversal, particularly perpendicular to the said relevantdirection is determined.

Still according to a further embodiment, when determining the firstguess border line, this one is determined as a physiologically relevantline passing through the reliable points.

According to the present border tracking method the correct border isdetermined along a sequence of two dimensional or three dimensionalultrasound image data and the correct border for each image frame can bedisplayed overlaid on the displayed image frame as an highlighted linecharacterized by a color which is different from the grey-scale B-modeimage displayed.

By means of the above tracking method of the border of two dimensionalimages or three dimensional images of an object, it is also possible todetermine the border line velocity and thus the border velocity of theobject imaged. Once the displacement of the border line is known alongthe sequence of two dimensional or three dimensional ultrasound imagedata sets and the time interval between each two consecutive sets ofimage data are known, the component(s) of velocity in the direction ofthe transmural cut(s) can be estimated for each reference point by meansof a simple calculation. The complete velocity vector is determined byevaluation of the other component(s) of velocity, the total number ofcomponents being two for two-dimensional imaging and being three forthree-dimensional imaging.

Each missing component of velocity can be evaluated once again with themethod of transmural cuts, such that for each reference point, on eachimage frame of the sequence of image frames, a transmural cut consistingof a line which crosses the tracked point and is directed along thedirection where the additional component of velocity must be evaluated,typically orthogonal to the direction previous employed for tracking theborder, the pixels taken along each transmural cut line in each of theimage frames of the sequence of image frames are placed in columns forall instants at once in a two-dimensional space time representation. Theevaluation of the velocity component along the chosen direction iscarried out along the space-time image using a cross-correlationprocedure of the pixel column in the space-time image with the velocitybeing given by the ratio of the column-wise displacement of thecorrelation maximum and the time interval between the correspondingframes.

This method is identical of that employed for tracking the border withthe difference that only the frame-by-frame displacement is required andthe eventual time integration of said displacement to get the motion ofthe border is ignored. A different evaluation of the velocity vector canbe obtained by applying two dimensional-correlation techniques or aspecific optical flow technique particularly developed for ultrasoundimage data of moving objects. The velocity estimation method can becarried out in combination with the above disclosed method for trackingthe border of the imaged moving object. The method is an adaptation ofknown method so called OPTICAL FLOW methods, like a known method calledPIV that is used in fluid dynamics.

Thus in displaying the B-mode images the border tracked can be drawn asa line as disclosed above and the velocity vectors of the border takenat certain number of points of the border line are displayed as arrowshaving a different color as the border line and the direction of thevelocity vector and a length corresponding to the modulus of thevelocity vector in the image plane of the two dimensional imagedisplayed.

By means of the method according to the present invention a differentapproach for tracking borders of a moving object in two and threedimensional ultrasound imaging is provided where the borders are not“detected”, rather they are “tracked”, i.e. followed in time, startingfrom one reliable existing instantaneous trace that is commonly, but notnecessarily, manually drawn by the experienced operator over one singleframe. Using this approach all the ambiguousness that is present in apure detection approach is cleared, and the original trace is followedin time by searching the maximum likelihood over its neighborhood in thefollowing frames. The tracking technique for each single point isapproached using a method based on transmural cuts that is similar tothat introduced in the document PCT/IT02/00114 filed on Feb. 27, 2002.Afterward the velocity on the tracked borders is estimated on the basisof the same maximum likelihood between two consecutive frames.

The automatic tracking method disclosed here allows the tracking of aborder on a sequence of two-dimensional or three-dimensional images, andthe evaluation of the velocity vector field on such borders. Inprinciple, the border could be tracked on the basis of the velocityvector only; however, a tracking procedure is a result of the summation(time integration) of the estimated velocities and is prone to an errorgrowth in presence of small incorrect estimates. This approach reducesthe two- or three-dimensional tracking to a combination ofone-dimensional tracking problems along the single topological relevantdirection (typically the orthogonal to the border) that can be muchbetter controlled and made accurate. On the other hand, the accuratetracking result is employed to improve the estimates of the velocityvector.

The result of this procedure is the automatic definition of the borderdisplacement and velocity over all frames of a sequence of images,starting from the border traced on a single image. Eventually, theborder information will be used to evaluated some geometric properties,like volume, area, or sizes, of the organ. The border kinematics(tracking+velocity) allows an estimate of global quantities (likevolumes, lengths) as well as local phenomena (like rotations, strain) ina unique approach.

The method according to the present invention is firstly described withreference to a two dimensional case. A sequence of two dimensionalB-mode image frames is acquired. The frames are acquired atpredetermined time interval one from the other.

Consider a sequence of two-dimensional (2D) images; mathematically, thisis a three-dimensional (3D) information that is 2D in space and 1D intime. Consider that the images contain one organ/object or part of it,that changes its position and shape in time, of which organ we want totrace the border kinematics at all instants.

The method according to the present invention comprises a first stepwhich consist of tracing the border over one single frame by defining aborder line. A border is traced, manually or by another manual orautomatic procedure, over one arbitrary frame. Such border is thendefined as a sequence of N points, defined by their coordinate pairs(xi,yi) with i=1 . . . N in the two dimensional image plane. The resultof this step is illustrated in FIGS. 1 and 2. FIG. 1 illustrates animage of the left ventricle where the endocardial border points aretraced from one side of the mitral annulus to the other side of the samemitral annulus. FIG. 2 illustrates an example in which the border is aclosed one where the Nth point connects to the first one.

Referring now to a particular case like the one illustrated in FIG. 1,where the border has a staring and an ending point, the method accordingto the invention provides for a second step of tracking the mostrepresentative reference point of the border line drawn in the firstimage frame.

According to this step, the general topology of the object border isreproduced on all the images by tracking the motion of a fewrepresentative points. These are commonly the starting and final pointsof the border when this is an open one. In specific cases additionalreference points can be added to improve the first evaluation of theregion about which the border must be sought.

The displacement of the representative reference points along one ormore specific directions is evaluated. FIG. 3 shows the reference pointsfor a left ventricle (in long axis view) that are the starting and finalpoints of the originally traced border. In this case, the physiologysuggests to track the motion of these points in the directioninstantaneously orthogonal to the mitral plane (that is defined by thesepoints).

The tracking along a specified direction is performed by using themethod of transmural cuts as follow. A line crossing the wall, passingthrough the point, and directed along the physiologically appropriatedirection is drawn; in the case shown in FIG. 3 the appropriatedirection is orthogonal to the mitral plane. In general two orthogonaldirection can be employed. The pixels taken along the chosen directionline(s) are placed in columns, each column corresponding to one frame ofthe sequence of images. In this way the evolution along a line can berepresented for all instants at once in a two dimensional space-timerepresentation (sometime referred as M-mode representation) where oneaxis is the distance along the line and the other axis is the time. Anexample of such a representation is shown in FIG. 4.

In the case of poor images with a low signal to noise ratio the spacetime representation can be built using a line for the transmural cutwith a thickness larger than that of a single pixel and extracting theaverage value across such a thickness. The border tracking is thenperformed along the space-time image.

Tracking Along the 2D Space-Time Image

The tracking procedure is a procedure for following a border along onedirection in a two-dimensional image like that in FIG. 4 starting from aknown position at one instant.

Let us call x the horizontal direction and y the vertical direction, andindicate with xi, i=1 . . . N, where N is the number of columns in theimage. The tracking is given by determination of a discrete sequence ofreal numbers yi=y(xi), starting from a known point yk corresponding tothe columns xk.

This is a one dimensional tracking problem that can be solved withseveral possible standard methods. One method is reported here forgiving completeness to the whole invention that can, however, employalso different techniques for this specific task when suggested by thespecific imaging employed.

The displacement from the known point yk to the point yk+1 can beestimated by evaluating the cross-correlation between the entire columnat xk with the entire column at xk+1. The cross-correlation functionwill present a maximum, the position of the maximum gives the value ofthe vertical displacement required to maximize the similarity betweenthe two columns, therefore yk+1 is estimated by adding such adisplacement to yk. This procedure is repeated between all pairs ofnearby columns and the result is an estimate of the entire border yi,i=1 . . . N.

In this procedure it is convenient to employ windowing techniques thatavoid side effects given by the two ends of the finite size columns.When applicable, it is also convenient to make use of the periodicity ofthe signal along x in order to perform the method in Fourier space.

The first estimate is improved by applying the same procedurerecursively on increasingly reduced spatial width about the previouslyfound border.

This first estimate yi can be further improved. To this aim a subset ofthe image is extracted by taking a few points above and below the firstestimate yi, and a new image whose center corresponds to the sequence yiis generated. A snake procedure like the one described in Blake A.,Yuille A. Active Vision MIT press, 1992, is employed to follow, in thenew image, the image brightness level that passes through the fixedpoint yk. As a result the estimation of yi, i=1 . . . N is refined.

As it will become clear in the following description the trackingtechnique is a unique procedure that is common to different steps of themethod according to the present invention.

As applied to the above mentioned step of the method according to thepresent invention, the result of this preliminary tracking procedure isthe position and displacement, at all instants, of the mostrepresentative reference points along the predefined direction, or thevector combination when two directions are employed.

After this, all the other points of the original border are rescaled ateach instant in order to get, at each instant a topologically equivalentborder geometry. Typically, like in the example of FIG. 2, all thepoints are translated along the original curve.

This preliminary resealing procedure permits to keep the referencepoints always at the proper position in all the frames, and to rearrangethe other points so that the reference maintains the same meaning in allthe frames.

The present step of tracking the most representative reference pointssuch as the starting and ending point of a border line can be avoidedwhen the specific geometry does not require or have any representativereference point to be tracked a priori. One example where this step canbe avoided is given by the closed geometry in FIG. 2.

After having carried out the tracking of the most representativereference points if these points are present or in place of the saidtracking step the method according to the invention provides for afurther step consisting in the tracking of all the other reference pointon the border line drawn manually or automatically in the first step ona first two dimensional image frame of the sequence of image frames.

For each point, independently, the tracking along a specified directionis performed by using the method of transmural cuts as follow. A linecrossing the wall, passing through the point, and directed along thephysiologically appropriate direction is drawn, this operation is madefor each instant/frame of the sequence of image frames because thepoints are not fixed in time but they have been previously rescaled ateach instant accordingly with the instantaneous displacement of thereference points. In most cases, like in the case shown in FIG. 5, theappropriate direction is taken at each instant as orthogonal to therescaled border. The pixels taken along each transmural line are placedin columns, each column corresponding to one frame of the sequence ofimages. In this way the evolution along a transmural cut, that is notfixed in all frames time but is slightly modified accordingly to therescaling, can be represented for all instants at once in atwo-dimensional space time representation analogous to that shown inFIG. 4.

In the case of poor images with a low signal-to-noise ratio the spacetime representation can be built using a line for the transmural cutwith a thickness larger than that of a single pixel and extracting theaverage value across such a thickness.

The border tracking is then performed along the space-time image usingthe same technique employed in the step of tracking the representativereference points and disclosed above in a detailed manner.

The result of this step is the position, at all instants, of all thepoints along the predefined direction, or the vector combination whentwo directions are employed. At this stage all the original points havebeen tracked in time, each one independently, and we have a new bordertracked over all frames.

It can be useful, especially in poor quality images, to improve theestimate by including a spatial coherence in the tracked border. Thiscan be done by verifying the likelihood of the tracking betweenneighboring points and correcting the eventual discrepancies withappropriate filters or validation methods.

As an additional procedure the method according to the present inventioncan be provided in combination with a procedure for determining theinstant border line velocity vector for each one of the reference pintsdefined on the border line as tracked on each two dimensional frame.

For each point, independently, the velocity vector can be known when twodirection (three for three-dimensional imaging) are employed fordisplacing it. When a single direction is employed, the completevelocity vector can be evaluated by selecting additional direction forthe transmural cuts on the already displaced point and evaluating thevelocity along the additional direction.

In the case of poor images with a low signal-to-noise ratio the spacetime representation can be built using a line for the transmural cutwith a thickness larger than that of a single pixel and extracting theaverage value across such a thickness.

Alternatively, the complete velocity vector can be evaluated by atwo-dimensional correlation technique or a specific optical flowtechnique adapted to the particular case of ultrasound imaging B-modedata. The two-dimensional result can then be improved by imposing itsaccordance with the previous estimate obtained for one component fromthe transmural cut approach. Results of the entire procedure are shown,for one frame, in FIGS. 1 and 2.

Three-Dimensional Imaging

The same steps described for the analysis of two-dimensional imaging canbe employed for the border tracking in three-dimensional imaging. Suchan extension is straightforward by using the previous steps in anappropriate combination, and substituting, in one case, the timedirection with one spatial direction. Eventually no additional manualintervention is necessary with respect to what is done intwo-dimensions, i.e. the indication of the border in one 2D frame.

A sequence of three-dimensional (3D) datasets is mathematically afour-dimensional (4D) information that is 3D in space and 1D in time.Consider that the images contain one organ/object or part of it, thatchanges its position and shape in time, of which organ it is desired totrace the border at all instants, the border now being a sequence oftwo-dimensional surfaces.

As a first step the method according to the present invention providesto choose one principal section plane which cuts to thethree-dimensional dataset, and to apply the entire two-dimensionaltechnique disclosed above on such plane.

The principal section plane of the 3D dataset is one plane, preferablyalong a physiologically relevant direction. Cutting the 3D datasets ofthe sequence of 3D datasets with this plane furnishes one sequence of 2Dimages.

FIG. 6 illustrates the cutting of a three-dimensional data set ofultrasound image data of the object 0 with two orthogonal principalsection planes 1 and 2 oriented in the vertical direction.

For each principal section plane the entire tracking procedure asdisclosed above for the sequence of two dimensional images is applied tothis two-dimensional sequence of images taken on the principal sectionplanes in order to track the border and evaluate the velocity on suchprincipal section plane. This border is the signature of the soughtborder kinematics on the plane.

The above steps can be repeated with more than one or two principalsection planes to improve the reliability of following steps in poorquality images.

After having carried out the above mentioned step a further step iscarried out consisting in defining secondary section planes to thethree-dimensional dataset, and applying the two-dimensional technique onsingle frames substituting the time direction with one spatialdirection.

The previous step allows to define the bounds of the surface border. Forthis, one direction is chosen over the plane cut used in the previousstep, preferably a physiologically relevant one (like the ventricleaxis), and, for each instant, evaluate the upper and lower bounds alongsuch direction of the border found in the previous step.

The range between these limits, at each instant, is further divided in Minternal points, and the 3D dataset is cut in correspondence of such Mpoints, with M secondary section planes that are orthogonal to thechosen direction as indicated with 3, 4, 5 in FIG. 7. By means of thesaid secondary section planes the corresponding M sequences of 2D imagesare constructed.

Successively, for each sequence, a reliable border in one single frameis defined, commonly the same frame used when the borders are drawnmanually during the previous step relative to the principal sectionplanes.

In each of such single M frames, the border now contains one or morereliable points, at the intersection with the principal section plane orplanes 1, 2 and that come from the border(s) determined in the previousstep relative to the principal section planes as illustrated in FIG. 8and indicated by R1, R2, R3, R4. A first guess border is constructed asa physiological relevant one passing through these reliable points R1,R2, R3, R4. An example of the said guess border on a secondary sectionplane is illustrated in the example of FIG. 9. Here the two dimensionalimage on a secondary section plane is illustrated together with the tworeliable points R1 and R2. The guess border passing through the said tworeliable points R1 and R2 is given by given by a circle in thetransversal images of the left ventricle.

A new border is now detected by the same procedure used for a singletransmural cut as disclosed in the previous chapter for the twodimensional case, this time however, substituting the time coordinatewith the spatial coordinate along such first guess border as follows.Make a number of transmural cuts on the single image along the guessborder, place the pixel found along each cut side by side in a newtwo-dimensional image and obtain a new image, like that in FIG. 4, wherethe horizontal axis does not indicate the time coordinate but thespatial coordinate along the tentative border. As a result the correctborder is tracked in one frame for each of the M sequences.

The above mentioned procedure is applied on all the sequences obtainedfrom the appropriate cutting of the three-dimensional dataset accordingto each secondary section plane defined. The tracking techniquedisclosed above in the previous chapter of the two dimensionalultrasound imaging is applied to each of the M sequences taking as astarting, reliable, border that found on one frame in the step relatingto the secondary section planes. The resulting M borders will define thecomplete surface border.

Similarly to the two dimensional case discussed above also in thethree-dimensional imaging case the instantaneous velocity vector for acertain number of predefined points on the border surface can becalculated by using the same technique disclosed of the two dimensionalcase. The two dimensional technique disclosed above is used here bysubstituting the two dimensional estimate with a three-dimensionalestimate of velocity.

When the tracking procedure is insufficient to define the entirevelocity vector, this is done by selecting additional direction for thetransmural cuts on the already displaced point and evaluating thevelocity along the additional direction.

Alternatively, this is done by using a three-dimensional correlation oroptical flow technique, in place of the two-dimensional one forevaluating the three-dimensional velocity vector.

While the preferred embodiment of the invention has been illustrated anddescribed in the drawings and foregoing description, the same is to beconsidered as illustrative and not restrictive in character, it beingunderstood that all changes and modifications that come within thespirit of the invention are desired to be protected.

1. A method of tracking position and velocity of objects' borders in twoor three dimensional digital images, particularly in echographic images,comprising the following steps: acquiring a sequence of at least twoconsecutive ultrasound image frames of a moving tissue or a movingobject, said ultrasound image frames being timely separated by a certaintime interval; automatically or manually defining a certain number ofreference points of a border of said moving tissue or object on a firstimage frame of said sequence of image frames; automatically trackingsaid border on at least one frame following said first frame bydetermining the new position of said reference points by estimating theposition of said reference points on the basis of the ultrasound imagedata of the acquired sequence of image frames, wherein the sequence ofimage frames acquired is a sequence of consecutive B-mode, grey scaleultrasound images; drawing either manually or by means of an automaticborder detection algorithm on a first frame a border line being definedby a trace of pixels of the image frame coinciding with said borderline, said trace of pixels coinciding with said manually orautomatically drawn border line is followed by searching the maximumlikelihood of said trace of pixels in said following image frame withsaid trace of pixels in said first image frame by analyzing the imagepixels in the neighborhood of said trace of pixels; tracking theposition displacements of one or more representative reference pointsover the entire sequence of consecutive image frames; rescaling theborder line drawn on said first image frame and at least on of saidfollowing image frames of said sequence of image frames according to thecorresponding position tracked of said representative reference points;defining a certain number of further reference points distributed alongsaid border line on said first image frame; tracking the position ofeach point independently from said other points along said sequence ofimage frames; tracking the position of said representative referencepoints and of said other reference points by defining a transmural cutline comprising a line which crosses said border line drawn and passingthrough said reference point; and wherein for images having a lowsignal-to noise-ratio, the space-time representation along saidtransmural cuts is built using a line for said transmural cut having athickness greater than that of a single pixel, and by extracting theaverage value across said thickness.
 2. A method of tracking positionand velocity of objects' borders in two or three dimensional digitalimages, particularly in echographic images, comprising the followingsteps: acquiring a sequence of at least two consecutive ultrasound imageframes of a moving tissue or a moving object, said ultrasound imageframes being timely separated by a certain time interval; automaticallyor manually defining a certain number of reference points of a border ofsaid moving tissue or object on a first image frame of said sequence ofimage frames; automatically tracking said border on at least one framefollowing said first frame by determining the new position of saidreference points by estimating the position of said reference points onthe basis of the ultrasound image data of the acquired sequence of imageframes, wherein the sequence of image frames acquired is a sequence ofconsecutive B-mode, grey scale ultrasound images; drawing eithermanually or by means of an automatic border detection algorithm on afirst frame a border line being defined by a trace of pixels of theimage frame coinciding with said border line, said trace of pixelscoinciding with said manually or automatically drawn border line isfollowed by searching the maximum likelihood of said trace of pixels insaid following image frame with said trace of pixels in said first imageframe by analyzing the image pixels in the neighborhood of said trace ofpixels; wherein a surface border is tracked in three-dimensional imagingby acquiring a sequence of three-dimensional ultrasound imaging datasets, each three-dimensional data set being acquired with apredetermined time interval from the previous data set; defining atleast a principal section plane for each of said three dimensional datasets along one chosen direction for obtaining a sequence of twodimensional image frames along said section plane; drawing a border lineof said imaged object either manually or automatically on the first twodimensional image frame of said sequence of two dimensional image framestaken along said principal section plane or each of said principalsection planes; tracking each sequence of two-dimensional image framesfor each corresponding principal section plane; defining for eachthree-dimensional data set of the sequence of three-dimensional datasetsa pre-established number of further secondary section planes crossingthe at least one of said principal section planes, said secondarysection planes being spaced apart one from the other along apredetermined direction and dividing said object represented by eachthree-dimensional data set of said sequence of three-dimensional datasets in slices; constructing for each secondary section plane in thesequence of said three dimensional data sets the corresponding sequenceof two-dimensional image frames relative to said secondary sectionplane; determining for each sequence of two-dimensional image framesrelatively to each secondary section plane a guess border line in onesingle frame, said guess border line passing across the intersectionpoints of the corresponding secondary section plane with the border linedrawn on said principal section plane; tracking said guess border lineby detecting a new border by way of substituting the time coordinatewith the spatial coordinate along said guess border line by defining acertain number of transmural cuts on said single image frame of saidsequence of images frames corresponding to each secondary section planealong said guess border line; identifying the pixels along saidtransmural cuts and placing said pixels along each of said transmuralcuts side by side for constructing a two-dimensional image where thehorizontal axis indicates the spatial coordinate along said guess borderline; and calculating the cross correlation between each of saidconsecutive pixel columns in said two dimensional image and thustracking said border line in one frame for each of said sequences oftwo-dimensional image frames corresponding to each of said secondarysection planes.
 3. A method according to claim 2 characterized in thatone or more further principal section planes can be defined.
 4. A methodaccording to claim 2 characterized in that two orthogonal principalsection planes are chosen with the crossing line of said two principalsection planes defining a preferred direction of said planes.
 5. Amethod according to claim 2, further comprising the following stepdefining bounds or limits for a distance range within which the group ofsaid secondary section planes has to be defined.
 6. A method accordingto claim 5 characterized in that a direction along which said secondarysection planes are distributed is chosen which is the same directiondefined for determining said principal section planes, along whichdirection bounds are determined for the ends of a distance range withinwhich the group of secondary section planes are defined and whichsecondary section planes are at least transversal to said direction. 7.A method of tracking position and velocity of objects' borders in two orthree dimensional digital images, particularly in echographic images,comprising the following steps: acquiring a sequence of at least twoconsecutive ultrasound image frames of a moving tissue or a movingobject, said ultrasound image frames being timely separated by a certaintime interval; automatically or manually defining a certain number ofreference points of a border of said moving tissue or object on a firstimage frame of said sequence of image frames; automatically trackingsaid border on at least one frame following said first frame bydetermining the new position of said reference points by estimating theposition of said reference points on the basis of the ultrasound imagedata of the acquired sequence of image frames, wherein the sequence ofimage frames acquired is a sequence of consecutive B-mode, grey scaleultrasound images; drawing either manually or by means of an automaticborder detection algorithm on a first frame a border line being definedby a trace of pixels of the image frame coinciding with said borderline, said trace of pixels coinciding with said manually orautomatically drawn border line is followed by searching the maximumlikelihood of said trace of pixels in said following image frame withsaid trace of pixels in said first image frame by analyzing the imagepixels in the neighborhood of said trace of pixels; determining theborder line instant velocity on each image frame of said sequence ofimage frames; determining a first estimation of the instant velocity foreach reference point on said border line by dividing the displacementvector of each of said reference points from a first to a followingimage frame of said sequence of image frames by the time intervaloccurred between said first and said following image frames; determiningthe instant velocity of said borer line for each of said image frames ofsaid sequence of image frames for each of said reference points on saidborder line by applying the method transmural cuts along the additionaldirection required to evaluate the complete velocity vector; whereinsaid complete velocity vector is determined by evaluating at least onecomponent of velocity along each direction, the total number ofcomponents being two for two-dimensional imaging and being three forthree-dimensional imaging, with each component of velocity vector beingevaluated with the method of transmural cuts providing the followingsteps: evaluating for each reference point, on each image frame of saidsequence of image frames, a transmural cut consisting in a line whichcrosses the tracked point and directed along the direction of anadditional component of velocity, said additional component typicallyorthogonal to the direction for said other components of said velocityvector; placing the pixels taken along each transmural cut line in eachof the image frames of said sequence of image frames in columns for allinstants at once in a two-dimensional space time representation; andcalculating said velocity component along the chosen direction along thespace-time image by using a cross-correlation procedure of the pixelcolumn in the space-time image, said velocity being given by the ratioof the column-wise displacement of the correlation maximum and the timeinterval between the corresponding frames.
 8. The method according toclaim 7 characterized in that it is identical of that employed fortracking the border except that only the frame-by-frame displacement isrequired and the eventual time integration of said displacement to getthe motion of the border is ignored.
 9. A method of tracking positionand velocity of objects' borders in two or three dimensional digitalimages, particularly in echographic images, comprising the followingsteps: acquiring a sequence of at least two consecutive ultrasound imageframes of a moving tissue or a moving object, said ultrasound imageframes being timely separated by a certain time interval; automaticallyor manually defining a certain number of reference points of a border ofsaid moving tissue or object on a first image frame of said sequence ofimage frames; automatically tracking said border on at least one framefollowing said first frame by determining the new position of saidreference points by estimating the position of said reference points onthe basis of the ultrasound image data of the acquired sequence of imageframes, wherein the sequence of image frames acquired is a sequence ofconsecutive B-mode, grey scale ultrasound images; drawing eithermanually or by means of an automatic border detection algorithm on afirst frame a border line being defined by a trace of pixels of theimage frame coinciding with said border line, said trace of pixelscoinciding with said manually or automatically drawn border line isfollowed by searching the maximum likelihood of said trace of pixels insaid following image frame with said trace of pixels in said first imageframe by analyzing the image pixels in the neighborhood of said trace ofpixels; determining the border line instant velocity on each image frameof said sequence of image frames; determining a first estimation of theinstant velocity for each reference point on said border line bydividing the displacement vector of each of said reference points from afirst to a following image frame of said sequence of image frames by thetime interval occurred between said first and said following imageframes; determining the instant velocity of said borer line for each ofsaid image frames of said sequence of image frames for each of saidreference points on said border line by applying the method transmuralcuts along the additional direction required to evaluate the completevelocity vector; and wherein for a combination of images of poor qualitywith a low signal-to noise-ratio, the space-time representation alongthe transmural cuts is built using a line for the transmural cut with athickness larger than that of a single pixel and by extracting theaverage value across said thickness.